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Air Traffic Control Sector 






Objective 


To identify and label instances where air-traffic controllers 
maneuvered aircraft in historical data 


Why Label Data 


1 . Improve future automation systems by understanding 
the way different controllers provide air-traffic control 
services 

2. Allows historical data to serve as a baseline for 
evaluation of future automation systems 


Outline 


• Trajectory prediction 

• Data source 

• Labeling controller interventions 

o Using outliers 
o Comparing time-series 

• Understanding trajectory prediction errors 

• Future Work 


Trajectory Prediction Systems 


• A predicted trajectory is a set of 4D points predicting 
where an aircraft will be in the future 

• Currently these predictions are used to: 

o Generate runway schedules for efficient arrivals 
o Help controllers identify aircraft-to-aircraft issues 

• Will serve as the foundation for future automation 
systems 
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Nobody is Perfect 


Predicted Trajectory 



Some Causes of Prediction Errors 


• Unknown wind speed 

• Unknown aircraft weight 

• Communication delays 

• Incorrect aircraft dynamic models 

• Unknown intent 


Change of Intent 
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Trajectory Prediction Behavior 





• Known intent changes: 


Two Types of Data 
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Error Components 

Along-track error: 


Cross-track error: 


Altitude errors: 


Predicted 




Data Sources 


• Aircraft radar tracks and filed routes are available 
through the FAA Aircraft Situation Display to Industry 
(ASDI) data 

• NASA can process track and route information using the 
Center TRACON Automation System (CTAS) to create 
predicted trajectories 


Simulated Data 


• For our analyses we used entirely simulated data 

• The Airspace Concept Evaluation System (ACES) was 
used to create both flown trajectories and predicted 
trajectories 

• The Advanced Airspace Concept (AAC) Autoresolver 
was used to emulate air-traffic controller behavior 

• Realistic errors were added to trajectory predictions 


Using Outliers 
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Looking at the Distributions 


Heavy tails indicative of times when errors are beyond 
standard levels 


Normal Q-Q Plots - One Minute Look-Ahead 
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Results Using Outlier Approach 

• Success of this approach relies on two assumptions: 

o Predicted trajectories are close to actual trajectory when no 
maneuvers are present 

o Maneuvers change trajectories significantly 

• Since these do not hold in all cases this method was 
only able to detect 70% of maneuvers without 
unreasonably high false-detection rates 


Identifying Intent Changes 


• Just looking at the error between predicted and actual 
points did not provide enough power to identify 
maneuvers 

• Instead we will look for differences between adjacent 
trajectory predictions 


Prediction at Time t 
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Time-Series Model 


• Fit a time-series model to a trajectory created at time t 
and predict what the data should look like at t=t n + 3 
minutes 

• Do this for all points in all trajectories 

• If the data at that point is greater than 2 standard 
deviations away from the forecasted point then this 
point is labeled a “singular point” 

• Remove singular points due to obvious geometric 
conditions like top-of-climb 


Identification Success Rate 


4000 


3000 


2000 


1000 


0 


Maneuver Identification 


2% Missed 

1 30% Missed 

25% Missed 

Horizontal Vertical Speed 



| Missed 
Identified 


Discussion of Results 


• Horizontal maneuvers are relatively easy to identify 

• Speed maneuvers were difficult to identify because 
prediction errors in speed overshadow these types of 
maneuvers 

• Vertical maneuvers may be missed due to coincidence 
with top-of-climb or top-of-descent points 


Understanding Error Distributions 
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Trajectory Geometry 
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Statistical Tools 
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Initial Distribution Analysis 

• Break trajectories down by flight phase 

o Before top of climb 
o Between top of climb and top of descent 
o After top of descent 

• Analyze the effect of look-ahead time 

• Determine the error variance for these factors 


Example Variance Calculation 
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Phase II Work 


• Improve the time-series method to improve sensitivity to 
altitude resolutions 

• Test identification algorithm on data from the real 
system 

• Identify causes of aircraft maneuvers and what type of 
maneuver is used in different situations 

• Add features to error analysis to more fully understand 
trajectory prediction errors 


